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SYSTEM AND METHOD FOR SMOOTHING SEAMS IN TILED 

DISPLAYS 

Field of the Invention 

[oooi] The present invention generally relates to the field of displays, and more 
5 particularly relates to multiple projector display systems. 

Background of the Invention 

[ooo2] Multiple projector systems have been proposed and used for many 
years. In the 1950s, the "CINERAMA" system was developed for the film 
industry. The CINERAMA system used three films to project three images using 
10 three separate projectors, which were then combined to form a single panoramic 
image. 

!ooo3] A difficulty with multiple projector display systems is that the multiple images 
often do not appear as one single continuous image on the display screen. When 
two images are projected side by-side on a single screen, there is normally a seam 

15 between the images. The final display image will either appear as two images 

placed side-by-side with a gap there-between or, if the images are made to overlap 
on a single screen, with a bright line there-between. Because of the inconsistencies 
in conventional cameras, video processing and delivery channels, displays and 
specifically projectors, it is exceedingly difficult to perfectly match the resultant 

20 video images so that no tiling artifact appears among the images. If the images are 
brought very close together on the same screen, there are typically overlaps at 
each seam regions. 

[ooo4] Therefore, systems and methods for providing generally seamless 
images in multiple projector display systems are needed. Furthermore, projector 
25 lens inherently distort images to some extent especially at the edges of the lenses. 
It is desirable that distortion in the displayed images be minimized, especially at the 
edges of the lenses, since this area of the lens is more likely to project images 
displayed closest to seams. 



SUBSTITUTE SHEET (RULE 26) 



WO 2005/067288 



PCT/US2004/044050 



2 

[ooo5] Conventional electronic distortion correction involves capturing, with a 
video image capture device, an image as it is displayed and generating feedback 
signals representative of the displayed distortion. Feedback signals are used to 
cancel out the distortion. Such a solution provides some improvement to the 
5 problem of lens distortion. However, among other drawbacks, this type of 
electronic distortion correction itself can add distortion, often adds to system 
complexity and cost, and complicates calibration and user interfaces. 
[0006] Therefore, systems and methods are needed for tiled display systems 
which do not include visually disturbing seams between the tiles. Further, there is a 
10 need for a tiled display system which merges together projected images from 
multiple tiles to form a tiled image, but which does not require high-quality (and 
expensive) optics. Therefore, a system that can electronically compensate for lens 
distortion is needed. Further systems that can adjust for lens distortion without 
relying solely on external capture of the displayed image are desirable. Further 
is systems and methods are needed that, provide a broader viewing angle than 

existing systems, while, at the sametime, more effectively concealing seams in tiled 
images. 

Summary of the Invention 

20 

[ooo7] The present invention provides method for producing a substantially 
seamless video image on a display surface. The method comprises the steps of 
separately projecting at least a first and a second video image onto a display 
surface such that a seam is defined by overlapping portions of said first and second 
25 video images. Inside the seam, the brightness of the first video image and the 

brightness of the second video image are adjusted electronically so as to provide a 
uniform brightness profile. 

Brief Description of the Drawings 

[0008] Preferred embodiments of the present invention will be described below 
30 in more detail, with reference to the accompanying drawings, in which: 
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iooo9] Fig. 1 is a top view of a display system comprising multiple projectors 
according to an embodiment of the invention. 

looio] Fig. 2 is a front view illustrating projectors of a display device 
arranged in accordance with an embodiment of the invention. 
5 [ooiii Fig. 3 is a graph illustrating the relationship between projected image 
brightness for pixels from projected images according to an embodiment of the 
invention. 

[ooi2] Fig. 4 is a graphic illustration of the relationship between projected image 
brightness for two projected images in the area inside a seam of a system 
10 according to an embodiment of the invention. 

[ooi3] Fig. 5 illustrates a representative uniformity curve for a lens of a 
projector suitable for use in the present invention. 

Detailed Description 

15 

[0014] The quality of a projected image is a function of several characteristics, 
all of which can suffer distortion in the process of translation from video data to 
displayed image. For example, brightness distortion, also referred to as luminance 
distortion has may possible sources. For example, due to the design of the optics 

20 within the light engines and/or the lamps themselves, most projectors do not project 
at a constant luminance level across the entire screen. This non-uniformity in 
luminance detracts from the displayed image. The luminance non-uniformity of a 
projector and its associated lens can become more pronounced under certain 
conditions. For example, when a "composite" image is created by multiple 

25 projectors whose individual images are tiled together, e.g., in a 4 by 5 pattern, to 
form the composite image, the non-uniformity in luminance is often much more 
apparent. Luminance non-uniformities can also be created (or existing non- 
uniformities made more pronounced) when the displayed image is viewed at an 
angle to the display screen or surface. That portion of the displayed image that is 

30 closer to the viewer will typically appear with a first brightness, while those portions 
located further away will appear to have a different brightness. 
[0015] Figure 5 illustrates an example of luminance uniformity of a light projected 
onto a screen 515 by a single projection lens (not shown). In this example, the 
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graph defines a lens distortion profile for the lens projecting the light onto the 
screen. As can be seen from the graph, the projected light is brighter near the 
center 520 of the screen. This center area corresponds to the center of the 
projecting lens. The brightness of projected light drops off from the center of screen 

5 515 toward the outer edges 501 , 502 of the screen. These areas correspond to the 
outer regions of the projecting lens used to project the light onto the screen 515. 
Each projecting lens has a corresponding brightness uniformity profile. According to 
one embodiment of the invention for the lens comprising the system, each lens 
profile is measured, or otherwise characterized, and the resulting data stored in a 

10 memory accessible to the image processor (for example processor 170 of Fig. 1). 
[0016] Figure 1 illustrates a plurality of example projectors 105, 110, 115 and 
120 arranged according to an embodiment of the invention. An image source 175 
provides a video image data representing a video, scene, or other visual image, 
either moving or still, to be displayed on display 130. The image data is typically 

is provided in a variety of conventional television and video formats depending on the 
particular capabilities of the associated display. In a system of the invention, video 
image data 1 80 is provided to a processor 170. Video image data 180 includes 
pixel brightness data for pixels x of the image. Processor 170 receives the video 
image data 180 including the pixel brightness data for pixels x. Processor 170, 

20 using tiling techniques, allocates the video image data 180 among the plurality of 
projectors 105, 110, 115 and 120 such that each projector is enabled to project a 
discrete image which, when projected onto display 130 and combined with the 
discrete images from the other projectors, will form a whole image to be displayed 
on display screen 130. 

25 [0017] Therefore, each projector is provided with respective image data181, 
182, 183, 184 for projecting the corresponding discrete image onto display 130. 
Figure 1 illustrates a simplified example. Video data 180 is provided to processor 
170. Processor 170 allocates data 180 such that video data 181 is provided to 
projector 105, video data 182 is provided to projector 110, video data 183 is 

30 provided to projector 183 and video data 184 is provided to projector 120. 
Projector 105 projects an image 106 onto screen . 130 based on the video data 
181. Likewise, projector 110 projects and image 111 onto screen 130, projector 
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115 projects an image 116 onto screen 130 and projector 120 projects and image 
121 onto screen 130. 

[001 8] As can be seen from Fig. 1 , images 1 06, 1 1 1 , 1 1 6 and 1 21 overlap. 
Image 106 overlaps image 111 so as to define a seam region 107. Image 1 1 1 
overlaps image 1 16 so as to define a seam region 112. Image 116 overlaps 
image 121 so as to define a seam region 117. The pixel brightness data 
corresponding to seam regions 107, 112, 117, is indicated in Fig. 1 as xi, x2, x3 
and x4. Accordingly, a portion x1 of video data 181 corresponds to a portion of 
the video image 106 that lies within seam 107, that is, between seam edge 208 
and seam edge 209 of seam 107. Likewise, a portion x2 of video data 182 
corresponds to of the video image 1 1 1 that lies within seam 107, that is, 
between seam edge 208 and 209 of seam 107. The correspondence between 
video data, video image and seam applies to the remaining projectors and 
projected images. In order to simplify the discussion, only projectors 105 and 
110 along with seam 107 will be described in detail herein. However, it will be 
understood that the same principles apply to the other projectors as well. 
[0019] In the embodiment illustrated in Fig. 1 projectors 105, 110, 115 and 
120 are of the spatial light modulator (SLM) type. One example of a commercially 
available SLM type projector known as a DLPTM projector. DLPTM is a 
trademark of Texas Instruments. In the example of Fig. 1 display surface 130 is 
spaced from a plane 131 of the projectors 105, 110, 115 and 120 by about 96". 
[0020] Each of projectors 105, 1 10, 1 15 and 120 is configured so as to project a 
corresponding image 106, 1 1 1 , 1 16 and 121 upon display surface 130. In the 
example illustrated in Fig. 1 display surface 130 measures about 120" in width. 
Each of projectors 105, 1 10, 1 15 and 120 is configured so as project their 
corresponding images so as to substantially fit substantially the entire original 
video image of display surface 130 when projected side by side.. Accordingly, 
images 106, 1 1 1 , 1 16 and 121 overlap thereby defining seams 107, 112, and 117. 
In this example each seam 107, 1 12, and 1 17 measures about 10" in width. 
[0021] The light engine for each projector comprises any suitable technology, 
such as one or more Liquid Crystal Display (LCD) panels, Digital Light 
Processing (DLP) or Liquid Crystal on Silicon (LCOS). Nonetheless, those 
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skilled in the art will recognize that the present invention may be used, with other 
projectors, including those using other types of image generation technologies. 
[0022] Each projector 205, 210, 215 and 220 includes optical elements to properly 
prepare the incoming illuminations to illuminate the SLM such as a DMD Tm and 
project the outgoing image. The optical path of such devices typically comprises 
two segments including an illumination path and a projection path. The illumination 
path starts with a high-reliability, metal halide, short-arc lamp that illuminates the 
DMD™ . The light from the arc lamp passes into a rotating RGB color filter wheel.. 
An illumination relay lens magnifies the beam to illuminate the DMD Tm and form a 
telecentric image at the DMD™ . A Total Internal Reflection (TIR) prism enables 
the incoming light from the lamp to pass onto the DMD Tm , and back into the 
projection optics. Depending on the rotational state of the mirror (e.g.±10 degrees 
for on/off), the light from the DMD Tm is directed into the pupil of the projection lens 
(on) or away from the pupil of the projection lens (off). A multiple-element 
projection cell magnifies the image coming off the DMD Tm at the desired MTF, 
lateral color, and distortion. 

[0023] As is known to those of ordinary skill in the art display devices are 

characterized by inherent non linearities and therefore, providers of such devices 
typically provide at least one gamma correction table to be used in correcting the 
nonlinearities. In addition system designers sometimes develop additional gamma 
tables in accordance with particular characteristics of the devices comprising the 
system design. The non-linearities include spatial non-uniformity, color non- 
uniformity, and/or luminance non-uniformity, but may also include other known 
image artifacts or irregularities. 

[0024] Fig. 2 illustrates an embodiment of image displaying surface 230 
having a total image display area measuring about 140" in width. In one 
embodiment of the invention, the image displaying surface is a display screen 
having a screen gain of unity. In the embodiment shown, the display surface is 
bordered by a bezel 230 having a width of 2". The display surface and bezel are 
seated in a frame having a width of 3". It is important to note, the dimensions given 
herein represent only one example of a wide variety of possible configurations, 
display dimensions, frame dimensions etc. for display systems. 
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[0025] The invention is suitable for implementation in other configurations 
too numerous to mention but these will be readily apparent to the skilled artisan 
upon reading the specifications and description of the invention contained herein. 
[0026] In the embodiment illustrated, each of a plurality of projectors (for 
5 example projectors 105, 110, 115 and 120, illustrated, for example, in Fig. 1) 

projects, substantially simultaneously, a corresponding image onto image displaying 
surface 130. Each projected image defines a corresponding discrete 205, 210, 215 
and 220 upon image displaying screen 230. Viewed as a whole, images 205, 210, 
215 and 220 substantially fill image displaying surface 230 with a single video 
10 image, frame of video, or picture, to be displayed to a viewer. The technique of 
arranging projectors, image portions and image displaying surfaces to display 
images in this manner is referred to herein as "tiling". A drawback of tiling 
techniques is the resulting visible artifacts in the displayed image that occur in the 
regions of the seams, that is in the region where an image projected by a first 
15 projector, overlaps an image projected by a second projector. According to an 
embodiment of the invention a seaming unit comprising a processor and method 
are provided to reduce tiling artifacts in the seams of the displayed image. 
[0027] Returning now to fig. 1 , processor 170 implements the methods 
according to various embodiments of the invention. As previously described 
20 processor 170 receives video signals representing the desired displayed image 
from image source 175. According to one embodiment of the invention, artifacts 
due to seams 107, 112 and 117 are corrected in the following manner. For 
purposes of discussion only projectors 105 and 110 will be discussed. However, the 
method is the same for the remaining projectors. Processor 1 70 provides data x1 
25 for projector 105 such that image 106 turns to black at edge 208. At the same time 
processor 170 provides data x2 for projector 110 such that image 1 1 1 turns to black 
at edge 209. This method is illustrated in more detail in Fig. 3. wherein line 106 
represents the brightness of pixels comprising image 106 from edge 208 to edge 
209. Likewise, line 1 1 1 represents the brightness of pixels of image 1 1 1 from edge 
30 208 to edge 209. The pixels of Image 106 are at a level brightness at edge. 208, 
wherein the pixel brightness values begin to decrease until a black value is obtained 
at edge 209. The pixels of image 111 are black at edge 208 and increase to a level 
of brightness at edge 209. 
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[0028] Therefore, processor 170 adjusts the brightness of the portion of 
image 106 displayed in seam 107 and the brightness of the portion of image 
111 displayed in seam 107 in an inverse relationship. 

[0029] According to an embodiment of the method, pixels displayed in a, 

5 seam region have a pixel position represented by j. Each displayed pixel j in the 
seam comprises a first pixel X, from the corresponding pixel of the image 106 
projected by projector 105, and a second pixel X 2 from the corresponding pixel of 
the image 111 projected by projector 110. For projector 110, for example, 
brightness of successive projected pixels X, is increased from black at edge 208 to 
10 a value Y at edge 209. The value Y corresponds to a uniform brightness for the 
seam region. At the same time, for the image projected by projector 205, the 
brightness of successive pixels X 2 decreases from Y at edge 208 and reaches black 
at edge 209. The relationship between pixels Xi, X 2 , j and Y is expressed, according 
to one embodiment of the invention, as follows: 



15 



(1) xl 



J 



^ overlap 



J) 



(2) jc2 = 1 J - 

^ overlap )j 



[0030] Wherein Xi represents a pixel of the image projected by projector 105, X 2 
represents a pixel of the image projected by projector 110, and j is the pixel number 

20 in the seamed area of the displayed pixel comprising Xi , X 2 . Overlap represents 
seam area in number of pixels, and gamma corresponds to the gamma correction 
for the projector. As can be seen, there exists an inverse relationship between the 
brightness of the image projected by projector 205 and the brightness of the image 
projected by projector 210 in the region defined by the seam. 

25 [0031] However, even with this technique, a shadow effect is sometimes 

observed. This effect is particularly noticeable if a viewer looks at the screen from 
an oblique angle. Thus there appears a small amount of increased brightness at 
the seamed area. Compensation for this effect is dependent on screen gain. For 
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purposes of discussing one embodiment of the invention, a screen gain of 1 is 
assumed. For embodiments of the invention employing lower screen gains, the 
compensation need not be as high as and is adjusted accordingly. 
[0032] Furthermore, the shadow effect is more easily observed for projector 
5 lens having relatively wide field angles. Conversely, the effect is less noticeable 
for projector lens having relatively narrow field angles. Nevertheless, the 
brightness profile in the seam area is not always 100% uniform. Depending on 
projection lens field angle and screen gain factor, the brightness profile in the 
seam area varies from linear to Gaussian. For purposes of discussion, the 
10 uniformity of each projector according to an embodiment of the invention is 
assumed to be 80% with ±20° field angle as illustrated in Fig. 5. 
[0033] According to another embodiment of the method, pixels displayed in a 
seam region have a pixel position represented by j. Each displayed pixel j in the 
seam comprises a first pixel X, from the corresponding pixel of the image 106 
15 projected by projector 1 05, and a second pixel X 2 from the corresponding pixel of 
the image 1 1 1 projected by projector 110. For projector 110, for example, 
brightness of successive projected pixels Xi is increased from black at edge 208 to 
a value Y at edge 209. The value Y corresponds to a uniform brightness for the 
seam region. At the same time, for the image projected by projector 205, the 
20 brightness of successive pixels X2 decreases from Y at edge 208 and reaches black 
at edge 209. The relationship between pixels Xi , X 2 , j and Y is, in this embodiment 
quadratic and is expressed, according to one embodiment of the invention, as 
follows: 



25 
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Wherein x1 1 represents a pixel from said first image portion after processing, x2 
represents a pixel from said second image portion after processing, j represents 
the pixel number in the seam region corresponding to the location of overlapping 
pixels x1 and x2, overlap is the seaming area in number of pixels, and gamma is 

5 related to the gamma correction of the projector. 

[0034] To further compensate for distortion due to non linearities, the 
techniques described above are further adjusted in accordance with the uniformity 
profile of each projector. According to an alternative embodiment of the invention 
pixels x1 and x2 are adjusted based upon the uniformity curve of a correspodning 

10 projector. An example uniformity curve is illustrated in Fig. 5 

[0035] In non tiled, i.e., single projector, display systems, gamma correction 
is commonly employed to correct for distortion in displayed images due to, for 
example, inherent non linearities in lens, imagers, displays and other imaging and 
display system components. Gamma correction can substantially reduce the 

15 distortion and image artifacts attributable to the properties of the imaging devices 
and system components themselves. According to one embodiment of the 
invention, projectors 205, 210, 215 and 220 are of a type that includes such 
gamma correction capabilities. 

[0036] According to one embodiment of the invention a method of reducing tiling 
20 artifacts in seam regions relies on adjusting the gamma correction of the projectors 
in a particular way so as to reduce tiling artifacts. In one embodiment, a method for 
reducing artifacts in seams of tiled images comprises the steps of applying a first 
gamma correction to image portions within the seams and a second gamma 
correction to image portions displayed outside the seams. 
25 [0037] Non-linearities due to display transfer functions can be corrected by a 
digital lookup table, referred to as a gamma table. The gamma table corrects for the 
differences in gain in the transfer function. According to one embodiment of the 
invention, gamma values from such a gamma table are adjusted so as to achieve a 
uniform brightness for the combined images, and, in one embodiment of the 
30 invention, further adjusted according to the characteristics of each projector 
comprising the system. 

[0038] According to one embodiment of the invention, the first and second 
image portions are adjusted based on the uniformity profile of projectors 105 and 
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1 10 respectively. According to one embodiment of the invention, gamma correction 
is provided based upon differences in brightness characteristics between seam 
regions and non-seam regions of the projected image. 

[0039]The invention may be embodied in other specific forms without departing 
from the spirit or essential characteristics thereof. The present embodiments are 
therefore to be considered in all respects as illustrative and not restrictive, the 
scope of the invention being indicated by the appended claims rather than by the 
foregoing description and II changes which come within the meaning and range of 
equivalency of the claims are therefore intended to be embraced therein. 
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